Change summary

Project layout

General cleanup to match best practice recommendations on project structure:

• file hierarchy to match sdsc-ordes/repository-template convention inspired by pitchfork and kriasoft conventions
  • bench moved under examples/{data,notebooks}
  • ad-hoc scripts (helpers, ci, dev, ...) go under tools
  • move all docker related files out of the package under tools/docker
• adopt src layout with pyproject.toml, following https://packaging.python.org conventions

Reusability

• the path to the shapes file is now read from an env variable instead of hardcoded
• a SHAPES_URL variable can be given to download the shapes on container startup
• remove shapes file from the git repo. Instead, the entrypoint downloads it dynamically on server startup
• add .example.env file for centralized overview of env variables
• remove specific mentions of the imaging plaza ontology

Modularity

The REST API and web application have been separated more clearly. This provides a smaller image with potentially fewer vulnerabilities for headless use-cases.

Currently: api image: 160MB, webapp image: 531MB

• webapp dependencies (streamlit) are now an optional dependency group in pyproject.toml
  • install only api pip install .
  • install api+webapp pip install '.[webapp]'
• Dockerfile is now multi-stage, with a api stage followed by a webapp stage.
  • This allows to build two separate images, one with only the rest api (image:version) and one with the api + webapp (<image>:<version>-webapp)

Development tools

Adds minimal tooling for development:

• Optional dependency groups for development and testing
• A makefile with standardized recipes for operations:
  • docker image management
  • project setup (install + dev dependencies)
  • code formatting

Usage

Just type make to see what operations are available with help messages. The rest should be straight forward.

Notes

• The large diff in python files is due to running ruff format to ensure standardized formatting across source files.
• This does not use a venv due to limitations imposed by the upstream alpine base image (from topbraid).
  • streamlit depends on pyarrow, which cannot be installed with pip on alpine, it needed to be installed at system level with apk, making it impossible (afaik) to use a venv separated from the system.

Questions

• Do we really need the current tests? they directly test the shacl tool. I would suggest dropping them and testing it through the web server. (in a dedicated PR)